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APPARATUS AND METHOD FOR DETECTING ROAD WHITE LINE 
FOR AUTOMOTIVE VEHICLE 

BACKGROUND OF THE INVENTION: 

5 Field of the invention 

[ 0001 ] The present invention relates to a road white line 
detecting apparatus and method using a CCD (Charge Coupled 
Device) camera and, more particularly, relates to road white 
line detecting apparatus and method which are capable of 

10 accurately detecting a position of the white line without 
an erroneous detection of edges of a smear developed on 
the CCD camera as the road white line. 

Description of the related art 
[0002] Recently, in order to achieve an automatic drive 

15 of an automotive vehicle or in order to assist a vehicular 
driver's driving of the vehicle, a video image outputted 
from a camera mounted on a vehicle is processed to detect 
such a white line as described above or to extract an image 
of a preceding vehicle running ahead of the vehicle 

20 (so-called, a host vehicle) from the image outputted from 
the camera- However, although it is ideal that a luminance 
state is stable during such an image processing as described 
above, such a stable luminance cannot always be achieved 
depending upon a usage environment. 

25 [0003] For example , while the road white line is detected 
using the CCD camera mounted on a vehicle body, a strong 
light source is present within a photograph range due to 
a presence of a light source itself, such as the sun and 
a headlight from an opposing vehicle which is running toward 

30 the vehicle at an opposing lane adjacent to the traffic 
lane of the host vehicle and due to reflections of light 
beams on a surrounding construction (building) and some 
of other vehicles. Consequently, such a smear phenomenon 
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that white rays of light are extended from the strong light 
source occurs in a charge transfer direction of the CCD 
camera . 

[0004] There is a high possibility that an erroneous 
5 detection of the white line occurs since the smear on the 
photographed image has the similar feature as the white 
line while a detection of white line is carried out . This 
provides a problem in a vehicular system having a traffic 
lane missing alarm system or lane keep assistance function. 
10 Hence, an automotive industry has demanded that the road 
white line should accurately be detected without a mistake 
that the smear edge is the white line. 

[0005] A Japanese Patent Application First Publication 
No. Heisei 9-190537 published on June 2, 1998 exemplifies 
15 a previously proposed on-vehicle road white line detecting 
apparatus . 

[0006] In the previously proposed road white line 
detecting apparatus disclosed in the above -described 
Japanese Patent Application First Publication, such a fact 

20 that the smear is developed due to a saturation of a charge 
in the CCD camera and a pixel value thereof is nearer to 
a saturation level than that of the white line is utilized. 
Then, the previously proposed road white line detecting 
apparatus determines the presence of the white line only 

25 when the pixel value of the line edge is strong and the 
pixel value on the white line is smaller than a predetermined 
threshold value. 
SUMMARY OF THE INVENTION: 

[0007] However, since, the previouslyproposedroadwhite 
30 line detecting apparatus determines the presence of the 
white line under such a condition determination that the 
pixel value is smaller than the predetermined threshold 
value, the real white line is erroneously determined as 
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the smear and cannot be detected in such a scene that shadows 
of ceiling and building are present on the road surface 
from which the smear occurs or in such a scene of a large 
brightness difference that shadows and sunlight spots are 
5 mixed in a photograph range such an inlet or outlet of a 
tunnel . 

[0008] It is, hence, an object of the present invention, 
to provide white line detecting apparatus and method which 
are capable of accurately detecting a road white line 

10 position without such an erroneous detection that a smear 
edge developed on the CCD camera is the white line. 
[0009] According to one aspect of the present invention, 
there is provided a road white line detecting apparatus 
(11), comprising: a CCD camera (13) to photograph a road 

15 surface; an edge image generating section (17, S10) that 
generates an edge image from a road surface image 
photographed and outputted by the CCD camera; a smear edge 
detecting section (19, S20 through S100) that detects edge 
positions of a smear from a region of the generated edge 

20 image which is determined on the basis of a previously 
detected position of a white line on the road surface; a 
smear edge eliminating section (21, S110) that eliminates 
smear edges corresponding to the edge positions of the smear 
from the generated edge image; and a white line detecting 

25 section (23, S120) that detects the present position of 
the white line from the generated edge image from which 
the smear edges have been eliminated. 

[0010] According to another aspect of the present 
invention, there is provided a road white line detecting 
30 method, comprising: providing a CCD camera (13) to 
photograph a road surface; generating an edge image from 
a road surface image photographed and outputted by the CCD 
camera; detecting (17, S10) edge positions of a smear from 
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a region of the generated edge image which is determined 
on the basis of a previously detected position of a white 
line on the road surf ace ; eliminating (21, S110) smear edges 
corresponding to the edge positions of the smear from the 
5 generated edge image ; and detecting (23, S120) the present 
position of the white line from the generated edge image 
from which the smear edges have been eliminated. 
[ 0011 ] This summary of the invention does not necessarily 
describe all necessary features so that the invention may 

10 also be a sub -combination of these described features. 
BRIEF DESCRIPTION OF THE DRAWINGS: 
[0012] Fig. 1 is a schematic block diagram of a white 
line detecting apparatus in a preferred embodiment 
according to the present invention. 

15 [0013] Fig. 2 is an example of a photographed image for 
explaining a processing region in a photographed image 
carried out in the white line detecting apparatus shown 
in Fig.l. 

[0014] Fig. 3 is an operational flowchart for explaining 
20 an operation of the white line detecting apparatus shown 
in Fig.l. 

[0015] Fig. 4 is an example of the photographed image 
processed under an image processing carried out in the white 
line detecting apparatus shown in Fig . 1 . 
25 [0016] Figs. 5A and 5B are timing charts for explaining 
a detection of edge accumulated values carried out in the 
white line detecting apparatus. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT: 
[ 0017 ] Reference will hereinafter be made to the drawings 
30 in order to facilitate a better understanding of the present 
invention . 

[0018] Fig. 1 shows a schematic block diagram of a white 
line detecting apparatus 11 in a preferred embodiment 
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according to the present invention. 

[0019] In Fig. 1, a CCD (Charge Coupled Device) camera 
13 is mounted on a predetermined position within a passenger 
compartment so as to enable a photographing of an image 
5 along a vehicular forward direction. An image photographed 
by CCD camera 13 is output ted to an edge image generating 
section 17. The image photographed by CCD camera 13 has 
its two-dimensional coordinate system with a left uppermost 
corner as an origin, a positive direction of Y axis being 

10 a lowered direction from the origin and a positive direction 
of X axis being a rightward direction from the origin. A 
charge transfer direction of CCD camera 13 is assumed to 
be the positive direction of the Y axis on the image. 
[0020] If such a high light intensity source such as the 

1 5 sun is present , such an image as shown in Fig . 2 is generated . 
That is to say, a smear such that white rays are superposed 
from the sun ( the light source) on the image of a road surface 
in the charge transfer direction (Y axis direction) of CCD 
camera 13 is developed. 

20 [0021] A controller 15 includes, in terms of hardware, 
a CPU (Central Processing Unit) (or MircoProcessor Unit) 
to govern the whole apparatus thereof in accordance with 
a control program; a ROM (Read Only Memory) to store the 
control program; and a RAM (Random Access Memory) to 

25 temporarily store control data. Software modules execute 
an edge detecting process, a smear edge detecting process, 
a smear edge eliminating process , and a white line detecting 
process in accordance with the control program . It is noted 
that such execution contents as described above may be 

30 achieved by the hardware. 

[0022] Hereinafter, processing contents of each section 

of controller 15 will be described in details. 

[0023] An edge image generating section 17 inputs the 
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image from CCD camera 13 and detects longitudinal edges 
and lateral edges from the input image by an edge detection 
process such as a well known Sobel filter. 
[0024] A smear edge detecting section 19 detects a smear 
5 edge developed according to a high brightness light source 
from the edge in a region determined on the basis of a 
previously detected white line position. 
[0025] A smear edge eliminating section 21 eliminates 
smear edges corresponding to edge positions of the smear 

10 detected by smear edge detecting section 19 from the edge 
image generated by edge image generating section 17. 
[0026] A white line detecting section 23 derives a 
displacement by comparing present white line edge positions 
to a two-dimensional road model which is an coordinate 

15 transformation carried out from a three-dimensional road 
model using a camera posture parameter and estimates the 
white line positions and a posture of the photograph device 
by a method of updating the road model on the basis of the 
derived displacement. 

20 [0027] Next, an operation of the white line detecting 
apparatus in the preferred embodiment will be described 
with reference to a flowchart shown in Fig. 3. 
[0028] At a step S10, edge image generating section 17 
detects longitudinal and lateral edges from the inputted 

25 image supplied from CCD camera 13. Consequently, the smear 
edges are extended in a longitudinal direction within the 
edge image, as shown in Fig. 2. 

[0029] At a step S20, the coordinate calculation is 
carried out on the two adjacent regions of smear edge 
30 candidate point detection region 35 and smear edge 
verification region 34 as a calculation of coordinates of 
the edge image region to detect the smear. 
[0030] With reference to Fig. 4, a process of detecting 
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the position of the white line will be described below. 
[0031] The white line detection process is carried out 
on rectangular white line search regions 40, as shown in 
Fig. 4. 

5 [0032] Thewhite line position is represented as an X-axis 
coordinate value in a vertical Y-axis coordinate value in 
respective search regions. 

[0033] Suppose that a Y-axis coordinate of a lower side 
in a K-th white line search region 40 is SY k and the detected 

10 white line position is a left side (SLX k , SY k ) and a right 
side (SRX k , SY k ) . The number of white line search regions 
40 are n at its left side and at its right side , respectively, 
and Y coordinates of the left and right search regions are 
the same . A lowest point of the detected white line position 

15 is set as left side SLn (SLXn, SYn) and as right side SRn 
( SRXn , SYn ) . It is noted that a lowest point of the detected 
white line position 33 in an actual space is expressed, 
for example, as about 10 meters ahead of the vehicle and 
a lower side of the photographed image is expressed as about 

20 6 meters ahead of the vehicle. 

[0034] A smear edge candidate detection region 35 is a 
rectangular region having a total of four vertexes of two 
points, viz., PLH (LXn + Xoff, SYn) and PRH (RXn - Xoff, 
SYn) which are offset in X-axis coordinate direction to 

25 two points SLn (SLXn, SYn) and SRn (SRXn, SYn) representing 
lowest points of the white line position detection process 
and remaining two points PLL(LXn + Xoff, SYn + DY) and PRL 
(RXn + Xoff, SYn + DY) moved in the Y-axis direction by 
DY from the above -de scribed points, as shown in Fig. 2. 

30 [0035] Xoff is provided for the white line not to be 
included in smear edge candidate detection region 35 and 
has a length of, for example, approximately 10 pixels. 
[0036] DY is a length from the detected white line 



position to the lower side of the image screen and, for 
example, of about 20 pixels. 

[ 0037 ] A smear edge verification region 34 is a rectangular 
region having four vert exes of the two points PLH (LXn + 
5 Xoff, SYn - DY) and RPM (RXn + Xoff, SYn - DY) . 

[0038] The above -de scribed two regions are inside left 
and right white line edges located at both sides of the 
road surface immediately before the vehicle and regions 
which are not easy for the forward (preceding) vehicle 

10 running ahead of the vehicle or a surrounding building to 
be photographed and where are less in nose invasion during 
the detection of a white line position. 
[0039] Especially, since the smear edge candidate 

detection region 35 does not include the white line, the 

15 white line is not erroneously detected as the smear. 

[0040] Referring back to Fig. 3, the smear candidate 
detection process is executed at steps S30 through S110. 
[0041] At step S30, Y-axis positive directional 

accumulated values of longitudinal edges with respect to 

20 each X-axis coordinate are derived within the rectangular 
regions on the longitudinal edge image. The accumulated 
values indicate large extreme values at portions having 
longitudinal edges such as the smear and displayed road 
surface, as shown in Fig. 5A. 

2 5 [0042] At a step S40, a positive maximum value Vpmax of 
each of the accumulated values with respect to the respective 
X-axis coordinate values, a negative minimum value thereof 
Vmmin, and positive and negative average values thereof 
Ap and Am are derived within the rectangular regions on 

30 the longitudinal edge image. 

[0043] Since, in the longitudinal edge image, a left side 
edge of a high pixel ( gray level) value region on an original 
image is positive and a right-side edge thereof is negative. 
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each accumulated value also indicates positive or negative 
value. For the positive value, a maximum value Vpmax and 
average value Ap in term of edge intensity are derived. 
Then, as far as the negative value is concerned, minimum 
5 value Vmmin and average value Am are derived. 

[0044] At a step S50, controller 15 determines that the 
smear occurs in a case where maximum value Vpmax of the 
light intensity in the positive side is equal to or larger 
than positive average value Ap by a constant magnification 

10 value 7 or in a case where the minimum value Vmmin of the 
negative side is equal to or smaller than negative average 
value Am by constant magnification value 7 • The constant 
magnification value, for example, 7 =20. Otherwise, 
controller 15 determines that no smear occurs and the routine 

15 goes to a step S120 at which a white line detection process 
is executed. 

[0045] In a case where the smear occurs ( Yes at step S50 ) , 
the routine goes to step S60, their respective threshold 
values TA1 and TA2 at positive and negative sides are derived 

20 using the following equations (1) and (2) on the basis of 
positive and negative maximum and minimum values of Vpmax 
and Vmmin and average values Ap and Am at step S40. 
TA1 = a * (Vpmax + Ap) --- (1) 
TA2 = a* (Vmmin + Am) (2). 

25 [0046] In the equations (1) and (2), a denotes a value 
to separate an average value derived due to a convex and 
recess of the road surface from the extreme values caused 
by the smear edges and is determined so that first threshold 
values TA1 and TA2 can be interposed into a numerical value 

30 between a road surface edge intensity and a smear edge 
intensity. For example, a = 0.5. 

[0047] At step S70, at smear candidate detection region 
35, the coordinate values of the positive extreme values 
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which are equal to or larger than positive first threshold 
value TA1 and the corresponding positive accumulated values 
and those of the negative extreme values which are equal 
to or smaller (negatively larger) than negative first 
5 threshold value TA2 and the corresponding negative 
accumulated values are determined as X-axis coordinate 
values AXi and their corresponding accumulated value AVi. 
[0048] In the case of Fig. 5A, X-axis coordinate values 
AXI, AX2, and AX3 of the extreme values of smear edge 
10 candidate points and corresponding accumulated values AVI , 
AV2, and AV3 are determined. 

[0049] At step S80, threshold value TBi respectively 
corresponding to individual accumulated value AVi for each 
smear edge candidate point is calculated in accordance with 

15 an equation (3). 

TBi = jQ * AVi --- (3) 
[0050] It is noted that jS is derived on the basis of a 
statistical value of a ratio between the accumulated values 
in smear edge candidate detection region 35 in the smear 

20 edge intensity and the accumulated values in the smear edge 
verification region 34 and is, for example, j3 = 0.5. 
[0051] At step S90, controller 15 derives a Y-axis 
directional accumulated value BVi, for X-axis coordinate 
AXi of smear edge candidate points in the same manner as 

2 5 step S30 within the smear edge verification region 34 in 
the same way as derived at step S20. 

[0052] At step S100, controller 15 compares second 
threshold value TBi derived at step S80 with the Y-axis 
directional accumulated value BVi derived at step S90 within 
30 the smear verification region 34 and detects X-axis 
coordinate AXi as smear edge coordinate values . 
[0053] In Fig. 5B, AXI and AX 2 are detected as smear edge 
coordinate values having the accumulated values equal to 
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or larger than second threshold values of TBI and TB2 . 
[ 0054 ] Firstly, in the case of smear diving the influence 
on the white line detection, vertical edges are detected 
on both regions of smear edge candidate detection region 
5 35 and smear edge verification region 34. However, since, 
in the case of the displayed road surface, no such a long 
vertical straight line display on the photographed image 
is present between both areas, such a possibility as the 
erroneous detection of the road surface display can be 
10 reduced. 

[0055] Secondly, since each of the first threshold 

values is set between each average value of the corresponding 
accumulated value and each corresponding maximum (minimum) 
value, a part having a relatively strong edge intensity 

15 is detected, the influence on the white line edges due to 
a remarkable road surface convex and recess can be reduced. 
[0056] Thirdly, since threshold values during the smear 
edge verification are calculated from the accumulated value 
at the same coordinate value as smear candidate detection 

20 region 35, the smear developed uniformly over the respective 
regions can selectively be detected. 

[ 0057 ] Fourthly, since the accumulated value can be taken 
only for the candidate in the smear verification region 
34, the processing time can be reduced. 

25 [0058] Referring back to Fig. 3 , at a step SI 10 , the smear 
edge coordinate detected as described above falls within 
the white line search region 40 on the white line detection 
process . In Fig. 4 , if Y-axis coordinate in the rectangular 
region representing white line search region 40 of the k-th 

30 number white line region from the top white line search 
region is SY k -i, SY k , a value of " 0 " is written into a 
rectangular region with (AXi - DX, SY^) and (AXi + DX, 
SY k ) as the corresponding vertexes. It is noted that, for 
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example, DX is determined to clear the part present in the 
smear edge having the strong intensity of DX. The processing 
time can be shortened by clearing only white line search 
region 40. 

5 [ 0059 ] At step S120 , the white line detection is carried 
out from longitudinal and lateral edge image. The method 
of detecting the white line detection from the longitudinal 
and lateral edge image is exemplified by a Japanese Patent 
Application First Publication No . Heisei 9-190537 published 

10 on July 22 , 1997 . That is to say, basically, the displacement 
is derived by comparing the present white line edge position 
with two-dimensional coordinate transformed road model 
using a camera posture parameter from the three-dimensional 
road model and the white line position and a posture of 

15 the photographing device (CCD camera 13) are estimated by 
updating the road model on the basis of the displacement 
to estimate the white line position and posture of the 
photographing device. 

[ 0060 ] Since the smear edge is eliminated from the process 
20 of the step S110, the white line can be detected from the 
image photographed by CCD camera 13 without the erroneous 
detection of the white line positions, the white line can 
be detected from the image photographed by CCD camera 13 . 
[0061] In addition, in this method, since the road model 
25 has been used, the stable estimation can be carried out 
even if the white line edge portions are not derived for 
the whole white line search region 40 , the white line position 
can stably be estimated even if the smear edge portion is 
partially overlapped over the white line edge and is cleared . 
30 [0062] As described above, since the series of processes 
at steps S10 through S120 are repeated, the influence of 
the smear on the photographed image by CCD camera 13 in 
which the smear is developed can be reduced so that the 
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white line of road can highly be detected with a high 
reliability. 

[0063] As described above, the edge image is generated 
from the image photographed by CCD camera 13 and the edge 
5 position of the smear is detected from the edge image at 
the region determined on the basis of the white line positions 
previously detected. Furthermore, the smear edges 
corresponding to the edge positions of the smear are detected 
from the generated edge image and the white line position 

10 is detected from the edge image from which the smear edges 
have been eliminated. Hence, the white line positions can 
be detected without erroneous detection of the smear edges 
as the white lines and this can contribute to an improvement 
in detection accuracy of a white line. 

15 [ 0064 ] In addition , since the smear edge candidate points 
are detected with the detection region set in the other 
region than that included in the detection region of the 
smear edge candidate points and the position of smear edge 
point within the smear edge verification region is verified 

20 on the basis of the coordinate values of the detected smear 
edge candidate points in the smear edge verification region 
which abuts the smear edge candidate point detection region 
of smear edge candidate points and which has been translated 
in parallel in the reverse direction to the transfer 

25 direction of the charge of CCD camera 13, the detection 
accuracy of the smear edge can be improved. Consequently, 
the white line positions can be detected without the 
erroneous detection of the smear edges as the white line. 
[ 0065 ] Furthermore , when the edge positions of the smear 

30 is included in the white line search region during the 
detection of white line, the smear edge in the internal 
of white line search region (40) is eliminated so that the 
processing time related to the elimination of smear edge 
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can be shortened. 

[0066] Still furthermore, the first threshold value is 
calculated on the basis of the extreme value and average 
value on the edge accumulated values in the charge direction 
5 as CCD camera within the detection region of the smear edge 
candidate point. The coordinate of the extreme value of 
the edge accumulated value exceeding the first threshold 
value and the edge accumulated value is detected as the 
smear edge candidate point. Then, the second threshold 
10 value for the respectively corresponding to the edge 
accumulated values of the smear edge candidate points are 
calculated. 

[ 0067 ] If the edge accumulated value in the verification 
region for the smear edge candidate point is larger than 

15 the second threshold value, the coordinate of the edge 
accumulated value is detected as the smear edge position. 
Hence, the detection accuracy of the smear edge can be 
improved. Consequently, the white line position can be 
detected without erroneous detection of the smear edge as 

20 the white line. 

[0068] The entire contents of a Japanese Patent 
Application No. 2000-356565 (filed in Japan on November 
22, 2000) are herein incorporated by reference. Although 
the invention has been described above by reference to 

25 certain embodiment of the invention, the invention is not 
limited to the embodiments described above. Modifications 
and variations of the embodiments described above will occur 
to those skilled in the art in the light of the above teachings , 
The scope of the invention is defined with reference to 

30 the following claims. 



